The present invention relates to an image sensing apparatus such as a digital camera or digital video camera or the like, and an image processing method for the image sensing apparatus.
A conventional single-plate color digital camera is a lens-integrated system, and is generally incapable of exchanging a lens. For this reason, an optical low pass filter and infrared ray (IR) filter or the like are provided before a CCD, which generates image signals according to light passing through the lens, thereby achieving a certain degree of effectiveness in reducing moiré or pseudo colors (erroneous colors). However, in a case of a camera which allows lens exchange, an IR filter can be provided before a CCD by forming a thin film on a glass surface of the CCD. However, securing space for an optical low pass filter before a CCD causes to increase the size of the camera body. Furthermore, although such optical low pass filter can reduce moiré or pseudo colors to a certain degree, a problem arises in that spatial frequency of an image decreases, and as a result, the image loses exact-focused characteristic such as that obtained in silver chloride photographs. Because of the above reason, the importance of an optical system without an optical low pass filter is increasing.
Furthermore, even if such optical low pass filter is provided, in a case of a digital camera employing a single-plate CCD, pseudo colors are still generated when color interpolation is performed. As exemplified by the Bayer pattern, when a number of R (red) and B (blue) color components is less than that of G (green) color components, the gap between these pixels becomes large, and this causes generation of pseudo colors in color interpolation. Furthermore, a method of generating three color planes with a digital filter or the like is available as a conventional color interpolation. However, since the order of the digital filter is limited, original image data resolution cannot be sufficiently expressed.
In view of the above, conventionally proposed is to perform image processing, particularly color interpolation, disclosed in U.S. Pat. No. 5,373,322 or No. 5,629,734, to obtain high resolution image data. This technique is effective to reduce pseudo colors, but not effective for moiré.
Even with the conventional technique, isolated pseudo colors, generated particularly around small characters or the like, cannot completely be eliminated. In view of this, it is proposed to convert color space, e.g., from RGB to L*a*b*, by an application program operating in a computer, and perform processing such as filtering on a* and b* to remove pseudo colors.
However, even if such conventional technique is employed, pseudo colors in color interpolation cannot completely be eliminated. This is further described below.
As shown in FIG. 19, if vertical white lines are exposed to green (G) and red (R) lines of a CCD at pixel pitch and conventional color interpolation is performed, red (R) and yellow (Y) vertical stripes are obtained (in FIG. 19, hatching portion indicates black where data is 0). Similarly, as shown in FIG. 20, if vertical white lines are exposed to green (G) and blue (B) lines of a CCD at pixel pitch and color interpolation is performed, blue (B) and cyan (C) vertical stripes are obtained. Note that even if the stripes in FIGS. 19 and 20 are horizontal instead of vertical, the same results, i.e., red (R) and yellow (Y) stripes or blue (B) and cyan (C) stripes, are obtained.
Furthermore, as another pattern, if white pixels having a checker flag pattern shown in FIG. 21 are exposed to R and B of a CCD and color interpolation is performed, a magenta (M) image is obtained despite the original white color. Still further, if white pixels having a checker flag pattern shown in FIG. 22 are exposed to G of the CCD and color interpolation is performed, a green (G) image is obtained despite the original white color.
In order to remove such pseudo colors, the color space is converted from RGB to, for instance, L*a*b* and filtering is performed on each of the a* and b* color spaces by an application program operating in a personal computer. However, in the case of checker flag patterns shown in FIGS. 21 and 22, a low frequency image of green cannot be distinguished from that of magenta, making it impossible to determine pseudo colors. If the frequency band of color difference is forcefully limited for JPEG compression, a dull (blurred) image is obtained. As a result, although the level of pseudo colors is reduced, the pseudo color components are spread out to the peripheral pixels, causing problems.
Furthermore, if the image data is compressed according to JPEG or the like without performing above-described pseudo color removing processing, block noise or the like may be caused. Moreover, if the frequency band of color difference is forcefully limited for JPEG compression, an image becomes blurred. As a result, although the level of pseudo colors is reduced, the pseudo color components are spread out to the peripheral pixels, causing problems. In order to eliminate such pseudo colors, it is ideal to employ a three-plate camera using three image sensing devices for each color component. However, such construction increases the size and cost of the camera.